COMBINATORIAL METHODS: FROM 
GROUPS TO POLYNOMIAL ALGEBRAS 



Vladimir Shpilrain 

Abstract. Combinatorial methods (or methods of elementary transformations) came to 
group theory from low-dimensional topology in the beginning of the century. Soon after 
that, combinatorial group theory became an independent area with its own powerful tech- 
niques. On the other hand, combinatorial commutative algebra emerged in the sixties, after 
Buchberger introduced what is now known as Grobner bases. The purpose of this survey is 
to show how ideas from one of those areas contribute to the other. 

1. Introduction 

Let F = Fn he the free group of a finite rank n > 2 with a set X = {xi, ...,Xn} of free 
generators. Let Y = {yi, ...,ym} and Y = {yi, ...,ym} be arbitrary finite sets of elements 
of the group F. Consider the following elementary transformations that can be applied 
to Y: 

(Nl) yi is replaced by yiyj or by yjyi for some j ^ i ; 
(N2) yi is replaced by y~^ . 

(N3) yi is replaced by some yj, and at the same time yj is replaced by yi. 
It is understood that yj doesn't change if j 7^ i. 

One might notice that some of these transformations are redundant, i.e., are composi- 
tions of other ones. There is a reason behind that which we are going to explain a little 
later. 

We say that two sets Y and Y are Nielsen equivalent if one of them can be obtained 
from another by applying a sequence of transformations (N1)-(N3). It was proved by 
Nielsen that two sets Y and Y generate the same subgroup of the group F if and only if 
they are Nielsen equivalent. This result is now one of the central points in combinatorial 
group theory. 

Note however that this result alone does not give an algorithm for deciding whether 
or not Y and Y generate the same subgroup of F. To obtain an algorithm, we need to 
somehow define the complexity of a given set of elements, and then to show that a sequence 
of Nielsen transformations (N1)-(N3) can be arranged so that this complexity decreases 
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(or, at least, does not increase) at every step (this is where we may need "redundant" 
elementary transformations !). 

This was also done by Nielsen; the complexity of a given set Y = {yi, is just 

the sum of the lengths of the words yi, ...,2/m- We refer to [15] for details. 

Nielsen's method therefore yields (in particular) an algorithm for deciding whether or 
not a given endomorphism of a free group of finite rank is actually an automorphism. 

A somewhat more difficult problem is, given a pair of elements of a free group F, to 
find out if one of them can be taken to another by an automorphism of F. We call this 
problem the automorphic conjugacy problem. It was addressed by Whitehead who came 
up with another kind of elementary transformations in a free group: 

(Wl) For some j, every Xi, i 7^ j, is replaced by one of the elements XiXj, xJ^Xi, xJ^XiXj, 
or Xi. 

(W2) Xi is replaced by x~^ . 

(W3) Xi is replaced by some Xj, and at the same time Xj is replaced by x^. 

One might notice a similarity of Nielsen and Whitehead transformations. However, 
they differ in one essential detail: Nielsen transformations are applied to arbitrary sets of 
elements, whereas Whitehead transformations are applied to a fixed basis of the group F. 

Using (informally) matrix language, we can say that Nielsen transformations corre- 
spond to elementary rows transformations of a matrix (this correspondence can actually 
be made quite formal - see [17]), whereas Whitehead transformations correspond to con- 
jugations (via changing the basis). This latter type of matrix transformation is known to 
be more complex, and the corresponding structural results are deeper. 

There is very much the same relation between Nielsen and Whitehead transformations 
in a free group. 

Note also that Whitehead transformation (Wl) is somewhat more complex than its 
analog (Nl). This is - again - in order to be able to arrange a sequence of elementary 
transformations so that the complexity of a given element (in this case, just the lexicografic 
length of a cyclically reduced word) would decrease (or, at least, not increase) at every 
step - see [14]. 

This arrangement still leaves us with a difficult problem - to find out if one of two 
elements of the same complexity (= of the same length) can be taken to another by an 
automorphism of F. This is actually the most difficult part of Whitehead's algorithm. 

In one special case however this problem does not arise, namely, when one of the 
elements is primitive, i.e., is an automorphic image of xi. If we have managed to reduce 
an element of a free group (by Whitehead transformations) to an element of length 1, we 
immediately conclude that it is primitive; no further analysis is needed. 
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Thus, the problem of distinguishing primitive elements of a free group is a relatively 
easy case of the automorphic conjugacy problem. As we shall see in Section 2, this is also 
the situation in a polynomial algebra. 

In Section 2, we review the results of various attempts to create something similar 
to Nielsen's and Whitehead's methods for a polynomial algebra in two variables. For a 
polynomial algebra in more than two variables, these problems are unapproachable so far, 
since we don't even know what the generators of the automorphism group of an algebra 
like that look like. 

In Section 3, we talk about retracts of a polynomial algebra in two variables. Basic 
properties of retracts of a free group are given in [15]. Since then, retracts have not been 
getting much attention until very recently, when Turner [21] and, independently, Bergman 
[4] brought them back to life by employing them in various interesting research projects in 
combinatorial group theory. Here we show the relevance of polynomial retracts to several 
well-known problems about polynomial mappings, in particular, to the notorious Jacobian 
conjecture. 

In the concluding Section 4, we have gathered some open combinatorial problems about 
polynomial mappings that are motivated by similar issues in combinatorial group theory. 

2. Elementary transformations in polynomial algebras 

Let Pn = K[xi, ...,Xn] be the polynomial algebra in n variables over a field K of 
characteristic 0. We are going to concentrate here mainly on the algebra P2. 

The first description of the group Aut{P2) was given by Jung [12] back in 1942, but it 
was limited to the case K = C since he was using methods of algebraic geometry. Later 
on, van der Kulk extended Jung's result to arbitrary ground fields. In the form we give 
it here, the result appears as Theorem 8.5 in P.M.Cohn's book [6]; this form is consistent 
with the idea of elementary transformations as described in the Introduction. 

Theorem 2.1. [6] Every automorphism, of K[xi, X2] is a product of linear automorphisms 
and automorphisms of the form xi xi + fix2); X2 X2. More precisely, if (51,52) is 
an automorphism of K[xi,X2] such that deg{gi) > deg{g2), say, then either {gi,g2) is 
a linear automorphism, or there exists a unique jjl G K* and a positive integer d such 
that deg{gi - jxgf) < deg{gi). 

The proof given in [6] is attributed to Makar-Limanov (unpublished) , with simplifica- 
tions by Dicks [8]. 

Note that the "More precisely, ..." statement serves the algorithmic purposes: upon 
defining the complexity of a given pair of polynomials (51, 52) as the sum deg{gi)+deg{g2), 
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we see that Theorem 2.1 allows one to arrange a sequence of elementary transformations 
(these are linear automorphisms and automorphisms of the form xi xi + f{x2); X2 — > 
X2) so that this complexity decreases at every step, until we either get a pair of polynomials 
that represents a linear automorphism, or conclude that (51, 52) was not an automorphism 
of K[xi,X2]- The parallel with Nielsen's method described in the Introduction is obvious. 

We also mention here another proof of this result (in case char K = 0) due to Ab- 
hyankar and Moh [1]. In fact, their method is even more similar to Nielsen's method in a 
free group. Many of their results are based on the following fundamental theorem which 
we give here only in the characteristic case (it will also play an essential role in our 
Section 3): 

Theorem 2.2. [1] Let u{t),v{t) e K[t] be two one-variable polynomials of degree n> 1 
andm > 1. Suppose K[t] = K[u,v]. Then either n divides m, or m divides n. 

Now let's see how one can adopt a more sophisticated Whitehead's method in a poly- 
nomial algebra situation. It appears that elementary basis transformations (see Theorem 
2.1), when applied to a polynomial p{xi,X2), are mimicked by Grobner transformations 
of a basis of the ideal of P2 generated by partial derivatives of this polynomial. To be 
more specific, we have to give some background material first. 

In the course of constructing a Grobner basis of a given ideal of P„, one uses "reduc- 
tions", i.e., transformations of the following type (see [2], p. 39-43): given a pair {p,q) of 
polynomials, set S{p, q) = fjj^ P — TT(qj ^' where l.t.{p) is the leading term of p, i.e., the 
leading monomial together with its coefficient; L = l.c.m.{l.m.{p),l.m.{q)) (here, as usual, 
I. cm. means the least common multiple, and l.m.{p) denotes the leading monomial oi p). 
In this paper, we'll always consider what is called "deglex ordering" in [2] - where mono- 
mials are ordered first by total degree, then lexicographically with xi > X2 > ■■■ > Xn- 

Now a crucial observation is as follows. These Grobner reductions appear to be of two 
essentially different types: 

(i) regular, or elementary, transformations. These are of the form S{p, q) = a ■ p — r ■ q 
or S{p, q) = a ■ q — r ■ p iov some polynomial r and scalar a G K*. This happens when 
the leading monomial of p is divisible by the leading monomial of q (or vice versa) . The 
reason why we call these transformations elementary is that they can be written in the 
form (p, q) {cap, ct2q) • where M is an elementary matrix, i.e., a matrix which 
(possibly) difi'ers from the identity matrix by a single element outside the diagonal. In 
case when we have more than 2 polynomials (pi, ...,pk), we also can write {pi,...,pk) 
{aipi, ...,akPk) • M, where M is a A; x A; elementary matrix; elementary reduction here 
is actually applied to a pair of polynomials (as usual) while the other ones are kept 
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fixed. Sometimes, it is more convenient for us to get rid of the coefficients and write 
(pi, {Pi, ■■■,Pk) ■ M, where M belongs to the group GEf-{Pn) generated by all 

elementary and diagonal matrices from GLk{Pn)- It is known [20] that GEj-{Pn) = 
GLk{Pn) if A; > 3, and GE2{Pn) GL2{Pn) if n > 2 - see [5]. 
(ii) singular transformations - these are non-regular ones. 

Denote by /(^(p) the ideal of P2 generated by partial derivatives of p. We say that a 
polynomial p E Pn has a unimodular gradient if /^(p) = P„ (in particular, the ideal /(^(p) 
has rank 1 in this case). Note that if the ground field K is algebraically closed, then this 
is equivalent, by Hilbert's NuUstellensatz, to the gradient being nowhere-vanishing. 

Furthermore, define the outer rank of a polynomial p G P„ to be the minimal number 
of generators Xi on which an automorphic image of p can depend. 
Then we have: 

Theorem 2.3. [18] Let a polynomial p E P2 have unimodular gradient. Then the outer 
rank of p equals 1 if and only if one can get from {di{p), d2{p)) to (1, 0) by using only 
elementary transformations. Or, in the matrix form: if and only if {di{p), d2{p)) ■ M = 
(1, 0) for some matrix M e GE2{P2). 

The proof [18] of Theorem 2.3 is based on a generalization of Wright's Weak Jacobian 
Theorem [22]. 

Remark 2.4. Elementary transformations that reduce {di{p), d2{p)) to (1, 0), can 
be actually chosen to be Grobner reductions, i.e., to decrease the maximum degree of 
monomials at every step - the proof [18] is based on a recent result of Park [16]. 

Now we show how one can apply this result to the study of so-called coordinate poly- 
nomials. 

We call a polynomial p E Pn coordinate if it can be included in a generating set of 
cardinality n of the algebra P„. It is clear that the outer rank of a coordinate polynomial 
equals 1 (the converse is not true!). It is easy to show that a coordinate polynomial has a 
unimodular gradient, and again - the converse is not true! On the other hand, we have: 

Proposition 2.5. [18] A polynomial p G P„ is coordinate if and only if it has outer rank 
1 and a unimodular gradient. 

Combining this proposition with Theorem 2.3 yields the following 

Theorem 2.6. [18] A polynomial p e P2 is coordinate if and only if one can get from 
{di{p), d2{p)) to (1, 0) by using only elementary Grobner reductions. 

This immediately yields an algorithm for detecting coordinate polynomials in P2 (see 
[18]) which is similar to Whitehead's algorithm for detecting primitive elements in a free 
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group. This algorithm is very simple and fast: it has quadratic growth with respect to 
the degree of a polynomial. In case p is revealed to be a coordinate polynomial, the 
algorithm also gives a polynomial which completes p to a basis of P2- 

In the case when K = <C, the field of complex numbers, an alternative, somewhat more 
complicated algorithm, has been recently reported in [9] . It is not known whether or not 
there is an algorithm for detecting coordinate polynomials in P„ if n> 3. 

Theorems 2.3 and 2.6 also suggest the following conjecture which is relevant to an 
important problem known as "effective Hilbert's Nullstellensatz" : 

Conjecture "G". Let a polynomial p G P2 have a unimodular gradient. Then one can 
get from {di{p), d2{p)) to (1, 0) by using at most one singular Grobner reduction. 

Remark 2.7. For n > 3, Theorem 2.3 is no longer valid since in this case, by a result of 
Suslin [20], the group = GEn{Pn) acts transitively on the set of all unimodular 

polynomial vectors of dimension n, yet there are polynomials with unimodular gradient, 
but of the outer rank 2, for example, p = xi + x\x2- The "only if" part however is 
valid for an arbitrary n > 2 - see [18]. It is also easy to show that one always has 
orank p > rank(Id{p))- 

Finally, we mention that our method also yields an algorithm which, given a coordinate 
polynomial p e P2, finds a sequence of elementary automorphisms (i.e., automorphisms 
of the form xi ^ xi + f{x2); X2 — > X2 together with linear automorphisms) that reduces 
p to Xi . 

3. Polynomial retracts 

Let K[x,y] be the polynomial algebra in two variables over a field K of characteristic 
0. A subalgebra R of K[x,y] is called a retract if it satisfies any of the following equivalent 
conditions: 

(Rl) There is an idempotent homomorphism (a retraction, or projection) ip : K[x,y] — »• 
K[x,y] such that ip{K[x,y]) = R. 

(R2) There is a homomorphism (p : K[x,y] R that fixes every element of R. 
(R3) K[x, y] = i? © J for some ideal I of the algebra K[x, y]. 

(R4) K[x, y] is a projective extension of R in the category of JC-algebras. In other words, 
there is a splitting exact sequence 1 — >/— >K[x,y] — >■ 1, where / is the same ideal 
as in (R3) above. 

Examples: K; K[x,y]; any subalgebra of the form K[p], where p € K[x,y] is a coor- 
dinate polynomial (i.e., K\p,q\ = K[x,y] for some polynomial q € K[x,y]). There are 
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other, less obvious, examples of retracts: if p = x + x^y, then K\p\ is a retract of K[x,y\, 
but p is not coordinate since it has a fiber {p = 0} which is reducible, and therefore is not 
isomorphic to a line. 

The very presence of several equivalent definitions of retracts shows how natural these 
objects are. 

In [7], Costa has proved that every proper retract of K[x,y\ (i.e., a one different from 
K and K[x,y\) has the form K\p\ for some polynomial p G K[x,y\, i.e., is isomorphic 

to a polynomial /-T-algebra in one variable. A natural problem now is to characterize 
somehow those polynomials p G K[x,y\ that generate a retract of K[x,y\. Since the 
image of a retract under any automorphism of K[x,y\ is again a retract, it would be 
reasonable to characterize retracts up to an automorphism of K[x^ y], i.e., up to a "change 
of coordinates" . We give an answer to this problem in the following 

Theorem 3.1. [19] Let K[p\ be a retract of K[x,y\. There is an automorphism ^ of 
K[x^y\ that takes the polynomial p to x + y ■ q iov some polynomial q = q{x,y). A 
retraction for K[tp{p)] is given then by x — > x + y ■ q; y ^ 0. 

Geometrically, Theorem 3.1 says that (in case K = C) every polynomial retraction of 
a plane is a "parallel" projection (sliding) on a fiber of a coordinate polynomial (which is 
isomorphic to a line) along the fibers of another polynomial (which generates a retract of 
K[x,y]). 

Our proof of this result is based on the Abhyankar-Moh theorem (see Theorem 2.2). 
Theorem 3.1 yields another characterization of retracts of K[x,y]: 

Corollary 3.2. [19] A polynomial p G K[x,y] generates a retract of K[x,y] if and only 
if there is a polynomial mapping of K[x, y] that takes p to x. The "if part is actually 
valid for a polynomial algebra in arbitrarily many variables. 

We also note that if a mapping described in Corollary 3.2 is injective, then p is a 
coordinate polynomial - this follows from the Embedding theorem of Abhyankar and Moh 

[!]• 

Theorem 3.1 has several interesting applications, in particular, to the notorious 

Jacobian conjecture. If for a pair of polynomials p,q £ K[x,y], the corresponding 
Jacobian matrix is invertible, then K[p, q] = K[x, y]. 

This problem was introduced in [13], and is still unsettled. For a survey and back- 
ground, the reader is referred to [3]. 

Now we establish a link between retracts of K[x,y] and the Jacobian conjecture by 
means of the following 
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Conjecture "R" . If for a pair of polynomials p,q E K[x, y], the corresponding Jacobian 
matrix is invertible, then K\p] is a retract of K[x,y]. 

This statement is formally much weaker than the Jacobian conjecture since, instead 
of asking for to be a coordinate polynomial, we only ask for p to generate a retract, 
and this property is much less restrictive as can be seen from Theorem 3.1. However, the 
point is that these conjectures are actually equivalent: 

Theorem 3.3. [19] Conjecture "R" implies the Jacobian conjecture. 

Another application of retracts to the Jacobian conjecture (somewhat indirect though) 
is based on the "(^°°-trick" familiar in combinatorial group theory (see [21]). For a poly- 
nomial mapping ip : K[x,y] —>■ K[x,y], denote by ip°°{K[x,y]) = r\'^^i'p''{K[x,y]) the 
stable image of (p. Then we have: 

Theorem 3.4. [19] Let (p he a polynomial mapping of K[x,y]. If the Jacobian matrix of 
(f is invertible, then either ip is an automorphism, or (p°°{K[x,y]) = K. 

The proof [19] of Theorem 3.4 is based on a recent result of Formanek [11]. 

Obviously, if (p fixes a polynomial p G K[x,y\, then p G (p°°{K[x,y\). Therefore, we 
have: 

Corollary 3.5. [19] Suppose <^ is a polynomial mapping of K[x,y] with invertible Jaco- 
bian matrix. If (p{p) = p for some non-constant polynomial p G K[x,y\, then ip is an 
automorphism. 

This yields the following promising re-formulation of the Jacobian conjecture: if is a 
polynomial mapping of i^[a;, y] with invertible Jacobian matrix, then for some automor- 
phism CK, the mapping a ■ ip fixes a non-constant polynomial. 

4. Some open problems 

In this section, we have gathered a few combinatorial problems about polynomial map- 
pings that are motivated by similar issues in combinatorial group theory. Two most 
important problems however - Conjectures "G" and "R" - appear earlier in the text (in 
Sections 2 and 3, respectively). 

Throughout, P„ = K[xi, ...,Xn] is the polynomial algebra in n variables, n > 2, over 
a field K of characteristic 0. 

(1) [10] Is it true that every endomorphism of P„ taking any coordinate polynomial to 
a coordinate one, is actually an automorphism? (It is true for n = 2 - see [10]). 

(2) Is there a polynomial p G Pn with the following property: whenever <p{p) = ip{p) 
for some non-constant-valued endomorphisms ip, ■0 of P„, it follows that ip = i/j? (In 
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other words, every non-constant-valued endomorphism of P„ is completely determined 
by its value on just 1 polynomial). 

(3) Suppose (p{p) = x\ for some monomorphism (i.e., injective endomorphism) ip of 
the algebra P„. Is it true that p is a coordinate polynomial? (It is true for n = 2 - see 
[19]). 

(4) Let p G P„ be a polynomial such that K\p] is a retract of P„. Is it true that 
ip{p) = xi for some endomorphism ip of the algebra P„ ? (It is true for n = 2 - see 
[19]). 

(5) Is it true that for any endomorphism if of the algebra P„, its stable image ip°°{Pn) 
is a retract of P„ ? (The answer to this question might depend on the properties of the 
ground field K). 
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